Non-volatile storage is essential to virtually all computer systems, from notebooks to desktops to large data centers employing clusters of servers. Non-volatile storage serves as a secure data repository which prevents data loss in the event of an unexpected interruption in primary power. Some common forms of non-volatile storage are packaged as non-volatile storage modules (NVSM) that can employ a magnetic disk (under control of a magnetic disk drive), flash memory components, or even magnetic tape (under control of a magnetic tape drive) as the non-volatile storage medium for the module.
One of the downsides of non-volatile storage is that it is relatively slow to access compared to volatile forms of memory such as DRAM (Dynamic Random Access Memory). Thus, virtually all computer systems also include volatile memory (VM) in which to temporarily store data for faster access. For example, code for executing application programs and data recently used by active applications are stored to and retrieved from the non-volatile storage and stored in the VM to increase execution speed. Volatile storage media such as semiconductor based DRAM, however, tends to be significantly more costly per unit of storage capacity then non-volatile memory such as magnetic disks.
Recently, a hybrid form of storage has been developed that seeks to provide the persistence of non-volatile storage but with an access speed comparable to VM. This form of storage is commonly known as a solid state disk (SSD). The SSD typically includes DRAM chips or some other form of VM and an NVSM that employs a non-volatile storage medium such as a magnetic disk, flash memory or the like. The SSD also typically includes a back-up or secondary power source such as a battery. The internal battery supply is used in the event that primary power is lost, preferably with sufficient capacity to continue refreshing the VM at least until all of the data stored therein is saved to the media of the NVSM. Once primary power is restored, the data can be retrieved from the NVSM and stored back into the VM for fast access by the host computer system to which it is coupled.
Because VM is substantially more expensive than non-volatile storage media, an SSD storage array endeavoring to provide storage capacity comparable to conventional nonvolatile storage arrays becomes very costly. Thus, if the amount of data that can be stored within the volatile storage of an SSD can be significantly increased without requiring a commensurate increase in the amount of physical memory employed for the VM, a significant cost/performance advantage would be realized. One technique that has been generally employed in memory systems to increase the storage density of a fixed amount of volatile memory is to subject the data to a data compression algorithm to reduce the volume of data prior to storing the data into the VM. If the data is of a type that is susceptible to substantial compression, the amount of data that can be stored within a given amount of volatile memory such as DRAM is increased without a requisite increase in the number of physical storage components.
A memory system employing compression can report to the host system within which it is deployed a “virtual” capacity that is typically n times its specified physical capacity, where n is equal to the numerator of a predetermined compression ratio the value of which can be specified based on, for example, the type of applications typically being executed by the system. Thus, if the predetermined compression ratio specified for the VM of a memory array is 3:1, and the amount of volatile memory is physically, 1 Gigabyte for example, the memory array can report a virtual capacity of 3 Gigabytes to its host computer system.
Serious issues can arise, however, when attempting to employ compression to increase the virtual capacity of a memory system's VM. Actual compression ratios at any instant in time will vary, depending upon the type of data that is being currently stored. For example, JPEG data is not as susceptible to compression as other types of data. Thus, if it becomes necessary during operation for the memory array to store a significant amount of JPEG data to a volatile memory array that has reported a capacity value based upon a certain predetermined average compression ratio, the short-term compression ratio can suddenly fall well below the predetermined average specified for the storage device.
If such a situation persists, the virtual capacity (and thus the currently available capacity) of the volatile memory of a memory system can quickly fall substantially below that which was initially reported to the system. Under such circumstances, the memory can exceed the currently available virtual storage capacity of the VM before the host system (such as through a memory array controller) can either divert the data to an alternate storage resource or at least to notify the system to stop sending it data. This can lead to the loss of data, which is particularly unacceptable in enterprise applications that most commonly employ SSD products.